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1.  EXECUTIVE  SUMMARY 


The  University  of  Wyoming  has  formed  a  robotics  initiative  consisting  of  three  distinct  parts.  A 
complete,  stand-alone  final  technical  report  is  presented  for  each  phase.  Phase  1  was  managed  by 
Dr.  Cameron  Wright,  Phase  2  by  Dr.  John  O’Brien,  and  Phase  3  by  Dr.  John  Mclnroy.  The 
overall  project  was  coordinated  by  the  Robotics  Initiative  Manager,  Dr.  John  Mclnroy. 

Phase  1,  “Biomimetic  Vision  Sensor,”  AFRL-RX-TY-TR-201 1-0096  summarizes  the 
development  of  a  novel  computer  vision  sensor  based  upon  the  biological  vision  system  of  the 
common  housefly,  Musca  domestica.  Several  variations  of  this  sensor  were  designed,  simulated 
extensively,  and  hardware  prototypes  were  constructed  and  tested.  Initial  results  indicate  much 
greater  sensitivity  to  object  motion  than  traditional  sensors,  and  the  promise  of  very  high  speed 
extraction  of  key  image  features.  The  main  contributions  of  this  research  include;  (1) 
characterization  of  the  image  information  content  presented  by  a  biomimetic  vision  sensor,  (2) 
creation  of  algorithms  to  extract  pertinent  image  features  such  as  object  edges  from  the  sensor 
data,  (3)  fabrication  and  characterization  of  sensor  prototypes,  (4)  creation  of  an  automated 
sensor  calibration  subsystem,  (5)  creation  of  a  light  adaptation  subsystem  to  permit  use  of  the 
sensor  in  both  indoor  and  outdoor  real-world  environments. 

The  pages  that  follow  constitute  a  complete,  stand-alone  final  technical  report  for  Phase  1 . 
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2.  INTRODUCTION 


AFRL  has  identified  a  need  for  more  eapable  vision  sensors  than  the  traditional  eharge  eoupled 
deviee  (CCD)  or  complementary  metal  oxide  semiconductor  (CMOS)  arrays.  In  particular,  there 
is  a  need  for  a  low-cost,  inexpensive  sensor  that  exhibits  fast  data  throughput,  is  extremely 
sensitive  to  even  tiny  object  movements,  is  relatively  unaffected  by  changes  in  ambient  light 
levels,  and  can  extract  pertinent  image  information  without  the  need  for  high-end  and  power 
hungry  computer  central  processing  units  (CPU).  The  biomimetic  vision  sensor  described  in  this 
report  is  being  developed  to  provide  these  capabilities.  Applications  for  such  a  sensor  would 
include  remotely  controlled  unmanned  ground  or  air  vehicles,  perimeter  security,  and 
autonomous  or  semi-autonomous  mobile  robots. 

Both  CCD  and  CMOS  focal-plane  array  sensors  are  commonly  used  and  perform  well  in  many 
imaging  applications,  but  have  certain  drawbacks  for  applications  that  require  high  sensitivity  to 
motion  and  high  speed  extraction  of  certain  image  features  such  as  object  edges  [1-5].  For 
example,  when  using  standard  CCD  or  CMOS  sensors,  objects  moving  at  high  speed  appear 
blurred  and  have  low  contrast,  objects  moving  less  than  a  pixel  across  the  array  appear  not  to 
have  moved  at  all,  and  if  edge  information  is  needed  it  requires  significant  computation  of  a 
computer’s  CPU. 

To  create  a  vision  sensor  that  overcomes  these  drawbacks,  a  biomimetic  (also  known  as 
“biologically  inspired”)  engineering  approach  was  embraced  to  take  advantage  of  proven 
biological  “designs”  found  in  the  animal  kingdom  and  then  adapt  salient  aspects  of  these  into 
more  capable  designs.  This  biomimetic  vision  sensor  is  based  on  physiological  aspects  of  the  eye 
(and  vision-related  neural  layers)  of  the  common  housefly,  Musca  domes tica.  The  fly  has  a  tiny 
brain,  yet  exhibits  visual  feats  that  exceed  the  best  traditional  computer  vision  systems.  This 
implies  that  a  biomimetic  vision  sensor  design  that  is  based  on  the  compound  eye  of  the  housefly 
could  eliminate  the  need  for  a  high  performance  computer  CPU  (the  “brain”)  in  the  vision 
system,  and  allow  greater  capability  on  certain  vision  tasks  (such  as  motion  detection)  than  is 
possible  with  traditional  sensors.  The  fly  eye  has  been  extensively  studied  over  many  years,  and 
the  research  presented  here  has  leveraged  this  large  body  of  knowledge  in  the  development  of  a 
biomimetic  vision  sensor  [6-30]. 

Such  a  biomimetic  “fly  eye”  vision  sensor  would  be  extremely  sensitive  to  object  motion,  even 
tiny  motions  that  standard  imaging  sensors  would  miss  entirely  (called  “motion  hyperacuity”  in 
the  literature  [10]),  and  be  able  to  extract  key  object  features  such  as  edges  at  very  high  speed 
with  no  CPU  involvement.  The  fly  exhibits  these  traits,  so  a  properly  designed  biomimetic  vision 
sensor  based  on  the  fly  should  also  exhibit  these  traits. 

The  fly  uses  a  combination  of  quasi-Gaussian  overlapping  photoreceptor  responses  and  neural 
superposition  to  achieve  what  has  been  described  in  the  literature  as  “hyperacuity,”  or  the  ability 
to  detect  image  features,  such  as  object  motion,  to  a  much  higher  degree  than  just  the 
photoreceptor  density  would  imply.  See  Figure  1  for  an  image  of  the  fly  eye,  and  Figure  2  for  an 
image  of  overlapping  Gaussian  responses. 
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Figure  1,  Scanning  Electron  Microscope  Image  olMusca  domestica,  with  a  Depiction  of  the 
Associated  First  Layer  Neural  “Wiring”  Superimposed  that  Shows  Neural  Superposition 

Image  courtesy  of  M.  Wilcox. 


Photoreceptor  Far-Field  Radiation 
R1  R2  R3 
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Figure  2,  Depiction  of  Overlapping  Gaussian  Response  Similar  to  that  Exhibited  hy  the 
Compound  Eye  of  the  Common  House  Fly,  Musca  domestica 
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Figure  3  shows  a  simplified  diagram  to  illustrate  this  quasi-Gaussian  (henceforth  called  simply 
“Gaussian”)  overlapping  photoreceptor  response  and  neural  superposition  that  both  contribute  to 
such  desirable  sensor  capabilities.  In  this  figure,  the  seven  shaded  photoreceptors  from  seven 
physically  distributed  ommatidia  (the  structural  elements  that  make  up  an  insect’s  compound 
eye)  shown  in  (a)  all  view  the  same  point  at  infinity.  The  overlapping  visual  field  of  the  three 
receptors  depicted  in  (b)  is  shown  as  (c).  All  seven  visual  fields  (as  half  power  radii)  are  shown 
perpendicular  to  the  optical  axis  in  (d).  Signals  from  the  physically  distributed  but  optically  co¬ 
aligned  photoreceptors  combine  in  the  neural  layer  and  together  are  called  a  “cartridge.”  While 
the  fly  has  eight  photoreceptors  in  each  ommatidium,  (Figure  1),  receptors  R7  and  R8  are 
“stacked”  and  share  a  single  optical  axis.  Note  that  the  Gaussian  overlap  for  an  artificial  sensor 
comes  from  the  front-end  optical  design,  whereas  the  neural  superposition  is  accomplished  by 
how  the  signals  are  subsequently  combined. 
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Figure  3.  Simplified  Diagram  Showing  Gaussian  Overlapping  Photoreceptor  Response  and 
Neural  Superposition;  Each  Hexagonal  Structure  in  (a)  is  Called  an  Ommatidium 


The  research  challenges  of  this  effort  were  to  better  understand  how  the  fly  eye  achieved 
capabilities  such  as  motion  hyperacuity;  to  create  an  optically  valid  but  low-cost  and  easily 
manufactured  design  approach  that  mimicked  the  salient  aspects  of  the  fly  eye;  and  to  develop 
algorithms  that  could  process  the  signals  from  such  a  sensor  in  order  to  extract  desired  image 
information  such  as  object  motion  or  object  edges. 
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3.  METHODS,  ASSUMPTIONS,  AND  PROCEDURES 


The  methodology  for  building  the  fly  eye  sensor  was  performed  in  a  building  bloek  approaeh. 

The  researeh  deseribed  in  this  report  is  a  eontinuation  of  an  earlier  projeet  that  was  performed 
under  ContraetNo.  FA4819-07-C-0003.  This  earlier  work  explored  the  physiologieal  aspeets  of 
the  fly  eye  that  were  pertinent  to  the  desired  eapabilities,  designed  the  opto-eleetronie  equivalent 
of  a  single  sensor  element  similar  in  many  ways  to  one  photoreeeptor  in  a  single  ommatidium  of 
the  eompound  eye  of  the  fly  (exhibiting  the  eharaeteristie  Gaussian  response),  investigated  the 
signal  eontent  of  sueh  a  sensor,  ereated  a  small  one-dimensional  array  of  sensors  with 
overlapping  Gaussian  visual  fields,  investigated  the  signal  eontent  of  the  ID  array,  and  devised 
algorithms  to  extraet  key  image  features  sueh  as  objeet  motion.  Based  upon  what  was  learned  in 
this  earlier  projeet,  the  eurrent  projeet  proeeeded  to  ereate  several  designs  for  a  two-dimensional 
array  eonstrueted  from  two-dimensional  versions  of  the  individual  sensor  elements.  Industry- 
standard  optieal  design  software,  Zemax,  was  used  to  explore  various  optieal  eonfigurations  that 
led  to  the  optieal  front-ends  of  the  hardware  prototypes. 

Extensive  simulation  was  performed  for  eaeh  sensor  design  to  ensure  the  desired  eharaeteristies 
of  motion  hyperaeuity  and  feature  extraetion  would  be  exhibited.  To  help  with  the  many 
simulations,  a  highly  flexible  simulation  software  framework  with  a  graphieal  user  interfaee 
(GUI)  was  ereated  for  a  wide  variety  of  sensor  parameters. 

Onee  the  simulation  results  were  satisfaetory,  hardware  prototypes  of  the  sensor  designs  were 
built.  Three  environments  were  used  for  testing  the  prototypes:  a  small,  highly  eontrollable  “light 
box”  where  the  ambient  lighting  eould  be  adjusted  easily;  a  large  “light  room”  with  white  fabrie 
on  all  walls  where  variations  on  typieal  indoor  lighting  eonditions  eould  be  introdueed  (both 
fiuoreseent  and  ineandeseent)  and  where  there  would  be  suffieient  room  to  move  target  objeets 
aeross  the  sensor  field  of  view  to  test  for  motion  and  edge  extraetion;  and  an  outdoor  area  where 
the  sensor  eould  be  exposed  to  mueh  more  ehallenging  but  realistie  lighting  seenarios  found 
outside.  Results  from  these  tests  are  deseribed  in  the  Results  and  Diseussion  seetion.  Additional 
subsystems  were  also  designed  for  the  sensors,  ineluding  a  49-ohannel  eustom  data  aequisition 
unit,  an  automated  sensor  ealibration  subsystem,  and  a  light  adaptation  subsystem. 

Early  results  of  the  outdoor  testing  eonfirmed  expeetations  that  the  dynamie  range  of  ambient 
lighting  from  indoor  to  outdoor  environments  exeeeded  eapability  of  a  linear  sensor  interfaee. 
Due  to  the  parallel  and  analog  nature  of  the  sensor,  methods  to  adjust  for  ambient  lighting  used 
in  traditional  CCD  or  CMOS  eameras  would  not  work.  Eor  example,  traditional  foeal  plane  array 
(CCD  or  CMOS)  eameras  eontrol  the  amount  of  light  ineident  on  the  array  by  adjusting  an 
aperture  in  the  single  eomplex  lens  system  and  eontrol  the  integration  time  of  the  light  by 
adjusting  a  meehanieal  and/or  eleetronie  shutter.  Sinee  the  fly  eye  sensor  is  based  on  a  eompound 
eye  with  many  small  simple  lenses,  having  individual  photodeteetors  for  eaeh  of  many  optieal 
axes,  an  aperture/shutter  approaeh  is  not  feasible.  The  faet  that  eaeh  of  many  optieal  axes  results 
in  many  ehannels  of  eleetronies  also  eonstrains  the  use  of  traditional  automatie  gain  eontrol 
(AGC)  eleetronies.  The  design  solution  to  this  is  deseribed  in  the  Results  and  Diseussion  seetion. 

One  assumption  made  was  that  the  sensor  design  eould  forego  the  physieal  separation  of  the 
photoreeeptors  that  make  up  a  biologieal  neural  eartridge  (Eigure  2),  as  long  as  the  optieal  axes 
aligned  eorreetly  and  the  visual  fields  exhibited  the  proper  amount  of  Gaussian  overlap.  This 
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assumption  would  eliminate  many  manufaeturing  difficulties,  and  it  was  hypothesized  that 
similar  visual  performance  could  still  be  obtained.  Another  assumption  made  was  that  all 
signal/image  processing  algorithms  used  with  the  sensor  must  consist  only  of  very  basic 
operations  such  as  multiplication,  division,  addition,  and  subtraction  since  that  is  all  that  the 
neural  interconnects  in  the  fly  eye  were  capable  of  performing,  and  that  adhering  to  this  would 
allow  for  the  implementation  of  the  signal  processing  in  low-cost  but  very  fast  parallel  analog 
hardware.  These  assumptions  were  validated  during  testing,  as  discussed  in  Section  4. 
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4.  RESULTS  AND  DISCUSSION 


The  following  summarizes  some  key  aspeets  of  the  results.  Note  that  this  work  is  a  follow-on  to 
an  earlier  eontraet,  so  this  section  builds  on  results  described  in  the  previous  report,  AFRL-RX- 
TY-TR-2009-4518. 

4,1,  Sensor  Designs 

Based  on  work  performed  in  the  earlier  contract  with  one-dimensional  sensors,  three  versions  of 
a  two-dimensional  sensor  were  designed;  one  mounted  on  a  planar  (flat)  surface,  one  mounted  on 
an  angled  surface  that  approximated  the  curved  surface  of  the  fly  eye,  and  one  on  a  truly  curved 
surface.  These  prototypes  are  listed  in  increasing  order  of  manufacturing  difficulty.  Figures  4-6 
depict  these  2D  sensor  prototype  variations.  The  angled  optical  front-end  shown  in  Figure  5  was 
an  intermediate  step  between  the  flat  sensor  and  curved  sensor  (middle  and  right  of  Figure  6, 
respectively).  As  such,  it  received  the  least  amount  of  testing. 


Figure  4,  Optical  Front-end  of  a  Sensor  Prototype  using  a  Planar  Surface  for  All  the 
Photodetectors;  This  Prototype  Includes  49  Optical  Axes 


Figure  5,  Optical  Front-end  of  a  Sensor  Prototype  using  a  Non-planar  Angled  Surface; 
This  Prototype  is  Intended  to  he  used  with  49  Optical  Axes 
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Figure  6,  Size  Comparison  of  Optical  front-ends:  (left)  First  Generation,  for  a  Maximum  of 
Seven  Optical  Channels,  Built  as  Part  of  the  Previous  Contract;  (middle)  Optical  Front-end 
of  the  49-axis  Sensor  shown  in  Figure  4;  (right)  Alternative  Optical  Front-end  using 
Optical  Fibers  and  a  Truly  Curved  Surface 


4.2.  Modeling  and  Simulation 

Simulations  were  performed  using  both  Zemax  and  MATLAB.  In  particular,  the  various 
tradeoffs  for  light  propagation  through  the  front-end  optics  were  investigated  by  simulating  with 
Zemax,  then  building  the  physical  optics  for  the  best  configuration.  The  simulations  related  to 
the  first  neural  layer  processing  of  image  features  such  as  edges,  motion  vectors,  and  such  were 
performed  with  MATLAB. 

These  simulations  provided  considerable  insight  into  the  potential  capabilities  of  a  biomimetic 
sensor  based  upon  the  fly  eye.  For  instance,  the  ability  to  easily  extract  edge  information, 
normally  a  computer-intensive  process,  was  shown  to  be  possible  using  simple  operations  that 
could  be  implemented  in  very  fast  parallel  analog  circuitry  in  an  actual  hardware  sensor  system, 
reducing  CPU  overhead  to  nearly  zero.  Similarly,  the  large-scale  simulation  model  showed  that  a 
fly  eye  sensor  should  be  able  to  extract  velocity  vector  (and  hence  flow  field)  information.  An 
example  of  this  is  shown  in  Figure  7,  where  six  ping  pong  balls  were  bounced  in  a  transparent 
container  and  the  simulation  provided  velocity  vectors  for  each  object  in  such  a  way  that  there 
would  be  nearly  zero  CPU  overhead  for  an  actual  sensor  system. 

The  smaller,  more  specific  computer  models  that  more  closely  simulated  the  various  sensor 
prototype  designs  confirmed  these  capabilities.  For  example.  Figure  8  shows  motion  detection 
and  velocity  vector  information.  The  simulated  objects  are  three  vertical  lines,  with  two  of  them 
moving  in  opposite  directions  (line  A  is  moving  to  the  right  fast.  Line  B  is  moving  to  the  left  at  a 
medium  speed),  and  one  (line  C)  is  stationary.  The  figure  shows  that  the  stationary  object  is 
ignored  while  the  velocity  vectors  of  the  two  moving  objects  are  extracted  correctly.  Note  the 
velocity  vectors  represent  the  entire  movement  from  the  U*  to  the  24*  image  frame. 
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Figure  9  shows  a  similar  result  for  simulated  point  objeets,  and  Figure  10  shows  the  result  of 
extracting  the  flow  field  information  for  a  simulated  object  approaching  the  sensor.  Once  again, 
note  the  velocity  vectors  represent  the  entire  movement  from  the  1®*  to  the  24*  image  frame. 


Figure  7,  Output  from  Large  Scale  Computer  Simulation  of  Fly  Eye  Vision  System 
Showing  Extraction  of  Object  Velocity  Vector  Information 
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Figure  8,  Extraction  of  Velocity  Vector  Information  for  Line  Objects 
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Figure  9,  Extraction  of  Velocity  Vector  Information  for  Point  Objects 
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Model  Results 
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Figure  10,  Extraction  of  Flow  Field  Information;  Simulation  Shows  the  Platform  on  Which 
the  Sensor  is  Mounted  Moving  Toward  a  Rectangular  Object  or  Opening 


To  better  standardize  testing  and  simulation  of  the  various  sensor  designs,  a  highly  flexible 
simulation  software  framework  with  a  GUI  was  created  that  could  be  used  to  easily  change  a 
wide  variety  of  sensor  parameters.  Some  of  the  user  interface  elements  of  this  software 
framework  are  shown  in  Figure  1 1  and  Figure  12. 
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Figure  11,  GUI  for  Changing  Many  Parameters  of  Fly  Eye  Sensor  Simulations 
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Figure  12,  GUI  for  displaying  Simulation  Results;  Buttons  at  the  Top  of  the  Display 
Window  become  Active  when  an  Image  is  Displayed 


The  software  framework  allows  the  user  to  easily  change  parameters  of  the  sensor  simulation. 
For  example,  the  comparison  between  a  hexagonal  pattern  of  photodetectors  and  a  “natural” 
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pattern  (i.e.,  closer  to  how  the  fly  photoreceptors  occur)  could  be  easily  accomplished.  These  two 
patterns  are  shown  in  Figure  13. 


Figure  13,  Two  Methods  of  Positioning  Optical  Axes  for  Photodetectors  for  the  Fly  Eye 
Sensor:  (left)  Hexagonal  Pattern;  (right)  “Natural”  Pattern 


4,3,  Hardware  Verification  and  Testing 

After  hardware  prototypes  were  constructed  for  the  biomimetic  fly  eye  vision  sensor,  the  electro- 
optical  properties  needed  to  be  verified  to  confirm  that  they  actually  approximated  the  salient 
aspects  of  the  fly  eye.  The  primary  aspect  of  interest  was  the  overlapping  Gaussian  profiles. 
Figure  14  shows  results  of  passing  a  line  object  past  the  sensor  (Only  three  optical  channels  are 
shown);  note  the  distinctive  Gaussian  shaped  overlap  between  individual  channel  signals.  The 
gain  in  the  electronics  had  not  been  adjusted,  so  the  heights  of  the  Gaussians  are  slightly 
different  in  the  figure;  this  was  easily  corrected  when  the  gain  settings  were  calibrated  later.  This 
figure  shows  many  results  at  once:  testing  both  indoors  and  outdoors,  use  of  light  adaptation 
described  later,  and  two  different  prototype  designs. 
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ILA  Response  to  Linear  Motion  (PdA) 


(a)  Linear  Circuit  PdA  Response 
OLA  Response  to  Linear  Motion  (PdA) 


ILA  Response  to  Linear  Motion  (OFA) 


(b)  Linear  Circuit  OFA  Response 
OLA  Circuit  Response  to  Linear  Motion  (OFA) 


Seconds 

(d)  Log  Circuit  OFA  Response 


Figure  14,  Test  Results  Showing  Desired  Gaussian  Overlap  in  Sensor  Array  Prototypes: 
ILA:  Indoors,  with  Light  Adaptation;  OLA:  Outdoors,  with  Light  Adaptation;  PdA: 
Photodiode  Prototype,  as  in  Figure  4;  OFA:  Optical  Fiber  Prototype,  as  in  Figure  6  (right) 


In  the  previous  eontract,  the  ID  sensor  array  prototype  was  used  to  compare  the  biomimetic 
vision  sensor  to  a  standard  CCD  camera.  It  was  investigated  how  well,  compared  to  the  CCD 
technology,  the  fly  eye  sensor  could  detect  fast  movement  and  also  very  tiny  movements  (i.e., 
translation  of  a  very  small  distance).  Traditional  CCD  sensors  exhibit  smearing  of  objects  that 
move  quickly  compared  to  the  integration  time  of  the  CCD  wells,  and  tiny  movements  that  do 
not  span  multiple  pixels  of  a  CCD  array  are  undetectable  to  traditional  sensors.  Testing 
confirmed  that  the  fly  eye  sensor  was  superior  to  the  CCD  camera  in  both  aspects.  It  was  shown 
in  the  previous  report  that  the  fly  eye  sensor  vastly  outperformed  CCD  arrays  for  responding  to 
fast  movements  and  tiny  movements,  even  in  very  low  contrast  scenarios.  The  results  for  CMOS 
arrays  would  be  very  similar.  For  this  report,  it  was  verified  that  these  same  characteristics  were 
present  in  the  2D  fly  eye  sensor  arrays.  Additional  hardware  verification  and  testing  is  described 
in  the  next  section  on  light  adaptation. 

4,4,  Light  Adaptation 

The  light  adaptation  subsystem  had  to  be  designed  specifically  for  a  compound  eye  type  of 
sensor.  As  discussed  in  the  Methods  section,  traditional  aperture/shutter  methods  of  controlling 
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light  are  not  practical  for  this  type  of  sensor.  The  hnal  design  of  the  light  adaptation  subsystem 
was  based  biomimeticahy  upon  the  “log  transform-subtraction-multiplication”  method  observed 
in  the  biological  fly.  For  each  7-channel  optical  axis  grouping,  this  was  performed  using  the 
circuit  shown  in  Figure  15. 
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Figure  15,  Light  Adaptation  Circuit  Implementing  a  “Log  Transform-Subtraction- 

Multiplication”  Process 
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Used  by  itself,  this  light  adaptation  circuit  reacts  almost  instantaneously,  which  is  detrimental  to 
the  sensor  performance.  Light  adaptation  is  meant  to  adjust  for  relatively  slow  changes  in 
ambient  lighting  to  allow  a  wider  dynamic  range  of  operation  than  would  otherwise  be  possible. 
It  was  found,  not  unexpectedly,  that  if  the  light  adaptation  is  allowed  to  be  too  fast,  it  will 
“adapt”  to  light  changes  due  to  objects  of  interest  instead  of  to  ambient  lighting,  and  make 
objects  of  interest  harder  to  detect.  The  solution  is  to  add  a  time  delay  to  the  light  adaptation 
subsystem.  Ah  biological  creatures  incorporate  such  a  time  delay  into  their  light  adaptation 
physiology. 

Traditional  electronic  time  delay  methods,  such  as  memory/shift  registers,  are  not  feasible  due  to 
the  need  to  preserve  the  analog  nature  of  the  signal.  A  method  such  as  ADC-delay-DAC  would 
be  too  expensive  and/or  introduce  too  much  quantization  error.  Older  analog  methods  such  as 
RC  or  RL  delay  lines  suffer  from  poor  stability  and  unrealistic  component  values  for  the  delay 
needed,  which  was  determined  to  be  approximately  one  second.  The  resulting  solution  includes  a 
sample  and  hold  circuit  with  a  timer  that  samples  the  analog  signal,  but  does  not  quantize  or 
digitally  encode  the  signal.  Thus  delay  was  achieved  without  compromising  the  analog  nature  of 
the  sensor  signal.  Circuits  associated  with  the  time  delay  are  shown  in  Figure  16  and  Figure  17. 
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Figure  16,  Absolute  Value  Circuit  used  in  the  Time  Delay  Subsystem;  This  Operation  is 
Sometimes  Referred  to  as  “Precision  Rectification” 
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Figure  17,  Time  Delay  Subsystem  used  in  Conjunction  with  the  Light  Adaptation  Circuitry 
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4,5,  Edge  Detection  and  Orientation 

The  sensitivity  of  the  2D  biomimetie  sensor  with  light  adaption  was  already  determined  to  have 
the  same  sensitivity  to  fast  motion  and  tiny  motion  deseribed  in  the  report  of  the  previous 
eontraet.  Its  ability  to  extraet  image  features  sueh  as  objeet  edges  and  edge  orientation  was  also 
tested.  It  is  important  to  recall  that  this  feature  extraction  is  performed  entirely  in  analog 
electronics,  and  at  very  high  speed,  with  zero  help  from  a  computer.  Figure  18  and  Figure  19 
show  the  sensor’s  ability  to  localize  a  horizontal  edge  of  an  object  in  its  field  of  view  for  indoor 
and  outdoor  environments. 


Indoor  Light  Adapted  Response  (PdA) 


(a)  Indoor  Light  Adapted  Response  PdA 


Differencss  (Indoor  PdA) 


(c)  Indoor  PdA  Difference  Response 
Horizontal  Edge  Lower  Location  (Indoor  PdA) 


(e)  Indoor  PdA  Location  Response 


Indoor  Light  Adapted  Response  (OFA) 


(b)  Indoor  Light  Adapted  Response  OFA 


Differences  (Indoor  OFA) 


(d)  Indoor  OFA  Difference  Response 
Horizontal  Edge  Lower  Location  (Indoor  OFA) 


(f)  Indoor  OFA  Location  Response 


Figure  18,  Horizontal  Edge  Localization,  Indoor  Environment 
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Outdoor  Light  Adapted  Response  (PdA)  Outdoor  Light  Adapted  Response  (OFA) 


(a)  Outdoor  Light  Adapted  Response  PdA 
Differences  (Outdoor  PdA) 


(b)  Outdoor  Light  Adapted  Response  OFA 
Differences  (Outdoor  OFA) 


(c)  Outdoor  PdA  Difference  Response 
Horizontal  Edge  Lower  Location  (Outdoor  PdA) 


(d)  Outdoor  OFA  Difference  Response 


(e)  Outdoor  PdA  Location  Response 


(f)  Outdoor  OFA  Location  Response 


Figure  19,  Horizontal  Edge  Localization,  Outdoor  Environment 


The  sensor  eorreetly  loealized  the  majority  of  objeet  horizontal  edges  presented  to  its  field  of 
view.  Results  for  vertieal  and  diagonal  edges  were  very  similar. 

4,6,  Scholarly  Outputs  from  this  Project 

As  part  of  this  project,  eight  students  completed  M.S.  degrees  in  Electrical  Engineering  (with  one 
pending  completion  of  requirements),  and  one  student  is  in  the  process  of  completing  the  final 
requirements  of  the  Ph.D.  degree  in  Electrical  Engineering. 
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A  total  of  16  papers  describing  this  work  were  published  and/or  presented  in  journals  and 
conferences  to  date  [31-46].  All  such  papers  were  subjected  to  peer-review  prior  to  acceptance. 
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5. 


CONCLUSIONS 


The  concept  of  a  biomimetic  fly  eye  vision  sensor  with  capabilities  that  improve  upon  traditional 
vision  sensors  was  validated  through  the  work  performed  on  this  contract.  The  most  pertinent 
conclusions  that  can  be  drawn  from  this  research  are  discussed  below. 

A  vision  sensor  based  upon  the  fly  eye  shows  the  ability  to  detect  object  movement  to  a  far 
greater  degree  than  traditional  vision  sensors  such  as  CCD  (or  CMOS)  arrays.  Fast  moving 
objects  exhibit  far  less  smearing  (and  its  associated  loss  of  contrast)  when  using  a  fly  eye  sensor. 
Very  slight  relative  object  movements  are  also  readily  detected  by  the  fly  eye  sensor,  whereas 
traditional  sensors  require  movement  that  spans  multiple  pixels  before  any  such  movement  is 
detectable.  This  is  termed  “motion  hyperacuity.”  The  fly  eye  sensor  requires  no  special  CPU 
processing,  and  does  not  require  multiple  image  frames  (as  do  many  “sub-pixel”  and  “hyper¬ 
resolution”  algorithms  intended  for  traditional  image  sensors). 

Detailed  simulations,  confirmed  with  hardware  prototypes,  show  that  with  the  fly  eye  sensor, 
important  image  information  such  as  object  edges,  velocity  vectors,  and  flow  field  information 
can  be  extracted  prior  to  any  CPU  processing  via  fast,  low-cost,  parallel  analog  electronics.  This 
has  multiple  benefits,  such  as  lower  CPU  overhead,  and  potential  reduction  in  image  data 
bandwidth.  All  hardware  test  results  shown  in  this  report  were  obtained  using  only  high  speed 
analog  circuitry;  zero  assistance  from  a  computer  was  needed. 

It  should  be  noted  that  from  an  optical  point  of  view,  the  overlapping  Gaussian  responses  which 
help  provide  capabilities  such  as  motion  hyperacuity  are  not  without  some  drawbacks.  In 
particular,  the  incoming  image  is  “pre -blurred”  to  a  certain  degree  in  order  to  achieve  the 
overlapping  effect.  Thus  traditional  measures  of  static  acuity,  such  as  the  ability  to  resolve  line 
pairs  in  a  test  target  image,  would  be  lower  using  the  fly  eye  sensor.  The  ramification  of  that 
issue  is  application  dependent,  but  the  general  conclusion  is  that  the  fly  eye  sensor  would 
probably  be  most  valuable  as  one  part  of  a  hybrid  vision  system.  As  part  of  the  hybrid  system, 
the  traditional  sensor  could  provide  higher  static  acuity  and  the  fly  eye  sensor  could  provide 
extreme  sensitivity  to  object  movements,  and  high  speed  extraction  of  edges,  and  flow  fields. 

The  potential  of  the  fly  eye  sensor  to  provide  vision  capabilities  that  are  far  greater  than 
traditional  sensors  can  provide  invites  many  exciting  application  scenarios  that  include  remotely 
controlled  unmanned  ground  or  air  vehicles,  perimeter  security,  and  autonomous  or  semi- 
autonomous  mobile  robots.  This  research  has  reached  the  threshold  of  significant  breakthroughs 
in  sensor  technology;  further  development  would  pay  great  dividends. 
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LIST  OF  SYMBOLS,  ABBREVIATIONS,  AND  ACRONYMS 


ID 

one-dimensional 

2D 

two-dimensional 

ADC 

analog-to-digital  eonverter 

AGC 

automatie  gain  eontrol 

CCD 

eharge-eoupled  deviee 

CMOS 

eomplementary  metal  oxide  semieonduetor 

CPU 

eentral  processing  unit 

DAC 

digital-to-analog  converter 

GUI 

graphical  user  interface 

OFA 

optical  fiber  array 

PdA 

photodiode  array 

RC 

resistive-capacitive 

RL 

resistive-inductive 
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